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Abstract 

In this paper we consider the communication problem that involves transmission of correlated sources over 
broadcast channels. We consider a graph-based framework for this information transmission problem. The 
system involves a source coding module and a channel coding module. In the source coding module, the sources 
are efficiently mapped into a nearly semi-regular bipartite graph, and in the channel coding module, the edges of 
O ■ this graph are reliably transmitted over a broadcast channel. We consider nearly semi-regular bipartite graphs 

as discrete interface between source coding and channel coding in this multiterminal setting. We provide an 
information-theoretic characterization of (1) the rate of exponential growth (as a function of the number of 
■ channel uses) of the size of the bipartite graphs whose edges can be reliably transmitted over a broadcast channel 

^*) | and (2) the rate of exponential growth (as a function of the number of source samples) of the size of the bipartite 

graphs which can reliably represent a pair of correlated sources to be transmitted over a broadcast channel. 

o 

00 , 

Q ! 1 Introduction 

°: 

! With the emergence of new set of applications such as wireless sensor networks, the problem of transmission of 
O ■ 

• • correlated information sources over multiterminal channels has received a renewed attention. In this problem, many 

> ■ 

correlated information sources are accessed by a set of transmitter terminals, and they wish to simultaneously 



' transmit some subset of them to another set of receiver terminals over a channel. In this paper we address the 
one-to-many communication system, where one transmitter terminal has access to all the information sources, and 
wish to transmit them to many receiver terminals. One such model involving two receiver terminals was considered 
by Han and Costa in £Q, and is described in the following. Consider a pair of correlated discrete memoryless 
sources (£>, T) with some generic joint distribution p{s, t) and a pair of finite alphabets S and T, respectively. The 
encoder observes long sequences of realizations of these sources (of length say n), and wishes to transmit them 
over a broadcast channel which has one input X and two outputs Y\ and Y2, and Receiver i has access to Yj. The 
channel behavior is governed by a generic conditional distribution p(yi, 1/2 \x). The channel is assumed to be discrete 
memoryless and is used without feedback. The encoder maps n-length source sequence pairs into n-length channel 
input sequences. Each receiver maps its corresponding n-length channel output sequences into its corresponding 
n-length source reconstruction sequences. The receivers would like to produce a reconstruction sequence pair such 

"This work was supported by NSF CAREER Grant CCF-0448115. This work was presented in part at the 39th Conference on 
Information Sciences and Systems (CISS), Baltimore, MD, March 2005, Information Theory and Applications workshop (ITA), San 
Diego, CA, February, 2006, and IEEE International Symposium on Information Theory (ISIT), Seattle, WA, July 2006. 
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that the probability that this deviates from the original pair goes to zero as blocklength n becomes large. If it is 
possible to build such sequences of mappings, then we say that the source pair is transmissible over the broadcast 
channel. The goal is to find the set of all sources that are transmissible over a given broadcast channel or the set 
of all channels over which a given source pair is transmissible. 

Two approaches have been proposed for this problem in the literature. On is called the joint-source-channel 
coding approach and the other is the separation-approach. The former approach 1 addresses the problem directly 
by finding the mappings for the given source pair and broadcast channel. A sufficient condition for transmissibility 
of a source over a channel has been given in yQ for this problem. In the separation-approach, we divide the encoding 
task into two sub-tasks, and similarly the decoding task is accomplished in two steps. In this approach, the n-length 
source pair is mapped into three indexes (referred to as Wo, W\ and W2) coming from three finite sets of size say 
Ao, Ai and A2, respectively. The first index (common message) is meant for both receivers, and the second and 
the third indexes (private messages) are meant for Receiver 1 and Receiver 2, respectively. This is called source 
encoding. The goal is to remove all the redundancy from the pair to produce three independent bit streams. Then 
these three indexes are mapped (referred to as channel coding) to n-length channel input sequences. On the other 
side of the channel, Receiver i first maps its n-length channel output into a pair of indexes (Wo and W) for i — 1,2. 
Then they independently produce source reconstruction sequences from the common and the private messages. The 
first goal is to find the set (called as the rate region) of all the rate tuples (i?o, Ri, R2), where Rj — ^logAj for 
j = 0,1,2, at which a reliable representation of the given source pair can be accomplished. The second goal is 
to find the set (called as the capacity region) of all rate tuples (Ro, R±, R2), at which a reliable communication of 
indexes over the given channel can be accomplished. The source coding part works under the assumption that the 
channel is noiseless, and the channel coding part works under the assumption that the messages are independent. 

The channel coding part by itself was first introduced by Cover in E] ■ The capacity region has been found for 
many interesting classes of broadcast channels |2"llHllHI51lr)ir7ll%ll??l 1101 ITTI 1121 llrtl I14j . The capacity region of certain 
class of broadcast channels used in wireless communication systems have been obtained in |15M16llTYl 1181 HUl 1201 121| . 
Marton [H] (also see E3E3EI1) established an inner bound to the capacity region for the discrete memoryless 
broadcast channel, which contains all the known achievable rate regions. Outer bounds to the capacity region have 
been obtained in |25l 1121 l2l)j . See [22] for a latest survey of the results on broadcast channels. The source coding 
part was addressed by Gray and Wyner in |28| . where a complete characterization of the rate region was given. We 
refer this source coding problem as Gray- Wyner problem. However, it is well-known that the separation-approach is 
not optimal for the one-to-many communication problem, unlike the case of point-to-point information transmission 
problem. In particular, a simple example was given in that showed that a triangular source can be sent reliably 
over a Blackwell channel by using a simple joint-source-channel coding scheme, but there is no way of transmitting 
that source over that channel using the separation-approach. 

Loosely speaking, in the separation-approach, there is an interface between the source coding module and the 
channel coding module. Due to the structure of the system, i.e., a common message and a pair of private messages, 
the interface can be thought of as a finite collection of products of finite sets. For example, in a system with 
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A,; denoting the size of the ith message for i = 0, 1,2, the interface can be thought of as U i= ° 1 (Ai x Bi), where 
Ai = {(i - l)Ai + 1, (i - l)Ai + 2, . . . ,iAi} and = {(i - 1)A 2 + 1, (i - 1)A 2 + 2, . . . ,iA 2 }. However, as seen 
above, this interface, a finite collection of products of finite sets, is not an efficient representation of a pair of 
correlated sources for transmission over broadcast channels 1 . Note that the corresponding interface for a similar 
approach in the point-to-point case is just a finite set. It is also well-known that a similar separation-approach is 
not optimal for other multiuser information transmission problems such as many-to-one communication |29| . 

In our recent work |3(J1 I31| , we have reported a bipartite graph-based framework for the problem of transmission 
of information in the many-to-one case. A similar approach was also studied in |32) . In the present work, we consider 
a similar approach to the one-to-many communication scenario. The fundamental motivation for this comes from 
the concept of typicality |33| . Given a correlated source pair (S,T), a sequence in S n is said to be typical (or 
individually typical) with respect to p(s), if its empirical histogram is close to p{s). Similarly one can define typical 
sequences in T" . A sequence pair in S n x T" is said to be jointly typical if its empirical joint histogram is close 
to p(s,t). Using the law of large numbers, it follows that (a) there are roug hly 2 nIi ^ and 2 nH ^ individually 
typical sequences in S n and T n , respectively, where H(-) denotes entropy (b) there are roughly 2 nH ^ s,T ^ jointly 
typical sequence pairs in 5™ x T", (c) for every typical sequence in <S™, there are roughly 2 nH ^ T]>s ^ typical sequences 
in T n that are jointly typical and vice versa, (d) probability, under p(s,t), of the set of jointly typical sequences 
(called jointly typical set) is close to 1, and (e) the probability, under p(s,t), of every jointly typical sequence pair 
is roughly equal to 2~ nH ( s > T \ These five properties lead one to associate a bipartite graph on the jointly typical 
set, with vertexes formed by individually typical sequences, and two vertexes are connected by an edge if they are 
jointly typical. Such bipartite graphs, where the degrees of the vertexes of one set is close to one constant, and 
the degrees of that of the other set is close to another constant, are referred to as nearly semi-regular Hence 
graphs can naturally capture the behavior of the source pair. The details regarding the source distribution can be 
dispensed with, and one can just work with this bipartite graph. This may also lead to the possibility of using them 
as discrete interface for one-to-many communication. The source encoder would now act on the source pair and 
produce correlated messages, or edges in a bipartite graph, and the channel encoder would now work with correlated 
messages and reliably transmit the edges in the graph over the broadcast channel. We would still have a source 
coding module and a channel coding module. However, now they would be interfaced using nearly semi-regular 
bipartite graphs rather than just a finite collection of products of finite sets. Of course, a finite collection of products 
of finite sets is a special case of nearly semi-regular bipartite graphs. 

We now present a brief summary of the results presented in this paper, for which we need some definitions. A 
nearly semi- regular bipartite graph is said to have parameters (Q\, 02,0'i,0' 2 ) if the ith vertex set has size nearly equal 
to 9i for i = 1,2, and the degrees of vertexes of the first set is nearly equal to 9' 2 and vice versa. With a slight abuse 

1 Another reason why this approach may be suboptimal is given in the following. In the characterization of the rate region in I28| . it 
turns out that for certain choices of the triple (Ro, Ri, R2) that belongs to the boundary of the rate region, i.e., optimal triple, the private 
indexes produced by the joint encoder will not be independent asymptotically. Hence, the channel coding module that follows, which 
works under the assumptions of independence, can not exploit this correlation, and is wasteful of resources. For example, in (15a) in 
1251 . if one chooses W such that (X, W, Y) do not form a Markov chain, then the triple (R ,R 1 ,R 2 ) = (I{X,Y;W), H(X\W), H(Y\W)) 
has this property. 
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of notation, a nearly semi-regular bipartite graph is said to have parameters (9q, 62, 8'i, 8' 2 ) if ^ is the union of 9q 
disjoint subgraphs each having parameters (8±, 82, 0[, 8' 2 ). A tuple of rates (Ro , R\ , R2 , R[ , R' 2 ) is said to be achievable 
for the given broadcast channel if there exists a bipartite graph with parameters (2 nRa , 2 nRl , 2 nR2 , 2 nRl , 2 ni?2 ) 
whose edges can be reliably transmitted by using the channel n times for large n. Similarly, a tuple of rates 
(i?o, Ri, -R2, R[, R'2) is sa id to be achievable for a given pair of correlated sources if there exists a bipartite graph 
with parameters (2 nRa , 2 nRl , 2 nR2 , 2 nRl , 2 nR - 2 ) which can reliably represent n realizations of the pair for large n. We 
provide information-theoretic partial characterizations of the sets of achievable tuples for a broadcast channel and 
a correlated source pair. These are presented in Theorem 1-4. Having the significance of the proposed framework 
mentioned first, let us look at the limitations of this optimistic framework as well. For that we need to look at the 
big picture. 

For the point-to-point case, to check the transmissibility of a source over a channel, we just need to check the 
non-emptiness of the intersection of two intervals [H, 00) and [0, C], where H denotes the entropy of the source, and 
C denotes the capacity of the channel. Note that only one parameter specifies the interval. For the one-to-many 
communication, the conventional separation-approach gives a sufficient condition for checking the transmissibility: 
non-emptiness of the rate region of the source pair and that of the broadcast channel. We need three parameters 
(i?o, R2) to specify the rate region in both source coding as well as channel coding. Clearly, a characterization 
that involves the fewest number of parameters is what we want. 

In the graph-based framework, we consider rate regions for the source and the channel, which are specified using 
five parameters: (Ro, Ri, R2, R'i, R' 2 )- However, the non-empty intersection of the rate region of the source pair with 
that of the channel still does not guarantee successful transmission. This is because, graphs having the same set of 
parameters may have different structures. It turns out that these graphs (that have the same set of parameters) 
can be partitioned into equivalence classes, where all graphs in an equivalence class have the same structure. This 
structure of the graphs has also been studied under the name of graph isomorphism in the literature. We will 
address this issue more formally in later sections. Hence a graph with parameters (say) (2 nRo , 2 nRl , 2 nR2 , 2 nRl , 2 nR - 2 ) 
which can reliably represent a source pair may not belong to the equivalence class of a graph with the same set 
of parameters whose edges can be transmitted reliably over the channel. In other words, to guarantee successful 
transmission of the source over the channel, we need to construct at least one pair of transmission systems (one 
for the source component and one for the channel component) for every equivalence class. Rather, in this work we 
have shown (a) the existence of an equivalence class for which a transmission system could be built in the source 
coding component, and (b) the existence of an equivalence class for which a transmission system could be built in 
the channel coding component. We plan to address this issue further in our future work. But we believe that the 
results given in this paper may be a first step toward an optimal discrete interface for multiuser communication. 

The outline of the remaining part of this paper is as follows. In Section |21 we first provide a summary of 
the results in a formal setting that are available in the literature that are closely related to our work. Then, in 
Section |3| we formulate the problem and consider certain properties of bipartite graphs that are relevant to our 
later discussion. Then channel coding part will be discussed in Section 01 resulting in an achievable rate region 
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for the broadcast channel with correlated messages. We also consider the special case of broadcast channels with 
one deterministic component. Thereafter, the complementary source coding part, the representation of correlated 
sources into graphs, will be described in Section After that, an example and its interpretation are provided in 
Section Finally, Section provides some concluding remarks. 

2 Preliminaries 

In this section, we provide an overview of the most important prior results in the literature on broadcast channels, 
and the related source coding problem, which are closely related to our work. 

2.1 Broadcast Channel with Independent private Messages 

A broadcast channel is composed of one sender and many receivers. The objective is to broadcast information 
from a sender to the many receivers. We consider broadcast channels with only two receivers since multiple re- 
ceivers cases can be similarly treated. Figure ^ shows a broadcast channel with one sender and two receivers. 
The discrete memoryless stationary broadcast channel consists of an input alphabet X and two output alphabets 
and and a conditional distribution p{y\,yi\x). The induced n-length conditional distribution is given by 
p{Vlt V% = nr=iP(yi*' V2i\ x i) when used without feedback. In other words, a broadcast channel is an ordered 
tuple (X , y±, y2,p(yi, V2\x))- We consider the following definitions for transmission of independent messages over a 
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Figure 1: Broadcast channel 



broadcast channel. 

Definition 1 A transmission system with parameters (n, Aq, Ai, A2, r) for the given broadcast channel {X, 3^i, 
y2,p{yi, J/2I2O) is an ordered tuple (/, 31,(72)7 consisting of one encoding mapping f and two decoding mappings g\ 
and <72 where 

• /:{l,2,...,A }x {1,2,..., Ax} x {1, 2, . . . , A 2 } -> X" , 
. 9l : X" -+{1,2,..., A } x {1, 2, ... , A,}, for 1 = 1, 2, 

• such that a performance measure given by the average probability of error satisfies: 

T = E E E AlA" Fr 92{Y 2 n )) + ((fc, i), (k, j))\X n = f(k, i, j)] . 

k=i t=i j=i 1 2 
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Definition 2 A rate tuple (Rq, R\, R 2 ) is said to be achievable for the given broadcast channel if for all e > 0, and 
for sufficiently large n, there exists a transmission system as defined above satisfying ^ log A, > Ri — e for i = 0, 1, 2 
and with the average probability of error r < e. 

Definition 3 The capacity region Cbci of the broadcast channel is the set of all achievable rate tuples (Rq, R±, R%). 

The capacity region of general broadcast channels is still not known. But Marton and Gelfand and Pinsker 
|13j have obtained an achievable rate region for the general discrete memoryless broadcast channel, which is the 
largest known inner bound to the capacity region. An achievable region of the discrete memoryless broadcast channel 
[3*5] is given by all rate tuples (Rq , R\ , R2 ) satisfying 

R <min{I(Z;Y 1 ),I(Z;Y 2 )}, (1) 

J2 + J2i</(Z,l7;y 1 ) ) (2) 

R + R 2 <I(Z,V;Y 2 ), (3) 

R +R 1 +R 2 < mm{I(Z; Yt),I(Z; Y 2 )} + I(U; Y X \Z) + I(V; Y 2 \Z) - I(U; V\Z) (4) 

for some p(z, u, v, x) on Zx U x V x X 1 where Z , U and V are auxiliary random variables with finite alphabets Z, 
U, and V, respectively such that (Z, U, V) — > X — > (Yi,Y 2 ) form a Markov chain. 

2.2 Gray-Wyner problem 

Consider a pair of correlated sources with a joint distribution p(s, t) and with finite alphabets S and T. The sources 
are assumed to be stationary and memoryless. These sources are denoted as an ordered tuple (S,T,p(s,t)). 

Definition 4 A transmission system with parameters (n, Aq, Ai, A2, r) for representing a pair of correlated sources 
(S,T,p(s,t)) is an ordered tuple (/o, /1, f 2 , gi, g 2 ) consisting of three encoding mappings f Q , fi and f 2 , and two 
decoding mappings g\ and g 2 where 

./,:5"xT^{l,2,..,A,}, for 1 = 0,1,2, 

• </a : {1,2,...,A } x {l,2,...,Ai}-fS» g 2 : {1,2,...,A } x {1,2,..., A 2 } — ► T n , 

• such that a performance measure given by the probability of error satisfies: 

r = ^[(. gi (/ (5",r"),/ 1 (5",r i )), ff2 (/ (5",r"),/ 2 (5",T")) + (s n ,T n )\ . 

Definition 5 A rate tuple (Rq, R\, R 2 ) is said to be achievable for the given correlated sources if for all e > 0, and 
for sufficiently large n, there exists a transmission system as defined above satisfying i log Aj < Ri + e for i = 0, 1, 2, 
and with the average probability of error t < e. 

Definition 6 The achievable region TZqwi for the correlated sources is the set of all achievable rate tuples. 
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An information-theoretic characterization of this rate region |281 136| is given in the following. For a given pair 
of correlated sources a tuple (i?o, -Ri, -R2) is achievable if and only if 

Ro >I(S,T;Z), (5) 
ifc > H(S\Z), (6) 
R 2 > H(T\Z), (7) 

for some distribution p{z, s, t) = p(s,t)p(z\s,t), where Z is an auxiliary random variable with a finite alphabet Z. 
Using convexity arguments, it can be shown that there is no loss of optimality if \Z\ < \S\\T\. 

This problem was also considered in |36| in a slightly different form. The minimum Rq that belongs to the 
rate region such that the corresponding Z satisfies T — ► Z — > S a Markov chain, is called as Wyner's common 
information C(S, T). 

2.3 Joint source-channel coding 

Consider the joint source-channel coding scheme studied in pp. Suppose we are given a pair of correlated sources 
without common part (SU OH OH! 2 an d a broadcast channel. 

Definition 7 A transmission system with parameters [n, r) for transmission of a pair of correlated sources (S, T,p(s,t)) 
and a broadcast channel (X, 3\, 3^2>p(yi> 2/2 1 a?) ) is an ordered tuple (/, 31,172) where 

• f :S n x T n -> X n 

• gi--y[ l ^S n , and g 2 : -> T" . 

• suc/i i/iaf a performance measure given by the probability of decoding error satisfies 

P n (s n ,nPr[(g 1 (Y{ l ) 1 g 2 (Y 2 n ))^(s n ,n\X n = f(s n ,n]. 

(s",i")£S"xT" 

Definition 8 A pair of correlated sources is said to be transmissible over a broadcast channel i/Ve > 0, and for all 
sufficiently large n, there exists a transmission system as defined above with parameters {n,T) such that r < e. 

The result of PP says that a pair of correlated sources is transmissible over a broadcast channel if 

H(S) < I(S,W,U;Y{) -I(T;W,U\S) (8) 
H(T) < I(T, W, V; Y 2 ) - I(S; W, V\T) (9) 
H(S, T) < min{/(VK; Y x ), I(W; Y 2 )} + I(S, U; Y X \W) + I(T, V; Y 2 \W) - I(S, U; T, V\W) (10) 

for some p(w, u, v, x\s, t) onWxMxVxIXiSxf, where W, U, V are auxiliary random variables with finite 
alphabets W, U and V such that (S, T) -> (WUV) -> X -» (Y 1 ,Y 2 ) form a Markov chain. 
This problem has also been considered in different settings recently in |39l I4()| . 

2 [T1 considered the general setting where sources may have non-zero common part. 
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2.4 An Example of transmission of Correlated Sources over the Broadcast Channel 

Let us consider an interesting example given in [I] showing the advantage of encoders that exploit the correlation 
between sources. Consider the transmission of a set of correlated sources (S, T) with the joint distribution p(s, t) 
given by 



\. s 

T ^\ 





1 





i 

3 





1 


1 

3 


1 

3 



with finite alphabet S = T = {0, 1} over a Blackwell channel with X — {1,2,3}, = 3^2 = {0,1} where the 
channel transition probabilities are specified by p(0, 0| 1) = p(0, 1|2) = p(l, 1|3) = 1. If we assign X=l, 2, and 3 to 
(S,T) — (0,0), (0,1), and (1,1), respectively, then Y\ and Y-x determine S and T without error, respectively. 

In the conventional separation-approach, first, factor (S n ,T n ) into three independent messages W\,W2, and 
Wo. Next, transmit (Wb,Wi) and (WbjWb) to receivers 1 and 2, respectively, over the broadcast channel using 
some, hitherto unknown, optimal coding scheme, so that (Wb,Wi) and (WcWb) reliably determine S n and T", 
respectively. Let the rate of Wi be Ri = —H(Wi) for % = 0, 1 and 2. Then, as the channel in consideration is 
deterministic, the sum of these rates must be bounded as R Q + Rx + R 2 < ^H{Y?, Y£) < ±H(X n ) < log 2 3 = 
H(S,T). 

According to the most efficient decomposition of this kind with the constraint Rq + i?i + R2 < H(S,T) is 
attained when Rq — C(S,T) In this case, 

Ro + Rt + R 2 = H(S, T) = log 2 3(bits) (11) 
2R + Ri+R 2 = log 2 3 + I(S, T; Z) > log 2 3 + C(S; T). (12) 

For this triangular source, C(S; T) = log 2 3 — /12(f) = § where h^ip) — ~ plog 2 p — (1 — p) log 2 (l — p). Therefore, 
2R Q + Ri + i? 2 > log 2 3 + § » 2.252 (bits). Consequently, R + Ri > 1 or R + R 2 > 1. Since H(Yi) < 1 
and H(Y 2 ) < 1 for any distribution, receiver 1 cannot reliably reproduce (Wo,Wi) or receiver 2 cannot reliably 
reproduce (Wo, W 2 ). Thus there is no way of reliably transmitting this triangular source via the Blackwell channel 
by factoring the sources into W\, W 2 , and Wo as shown above. 

3 Problem Formulation 
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Figure 2: Transmission of correlated sources over a broadcast channel 
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The problem we are addressing is the simultaneous transmission of two correlated sources (S,T,p(s,t)) over a 
broadcast channel {X , 3^, 3^ 2 ,p(?/i, U2\%)), with one sender and two receivers as shown in Figure|21 Here, the encoder 
can access both sources (S, T) and the receivers can not communicate with each other. The encoder is given by 
a mapping / : S n x T n — > X n . The decoders are given by mappings g\ : yf — > S n and g 2 : ~y% — * T n . The 
performance measure associated with this transmission system is the probability of decoding error: 



Pr[(S n ,T n ) ^ (g 1 (Y 1 n ),92(Y 2 n ))}. 



3.1 Basic Concepts 
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Figure 3: The outputs of sources are first mapped into edges in a bipartite graph, and the edges coming from this graph are 
reliably transmitted over a broadcast channel 



We consider a modular approach to this problem, which is shown in Figure |3J The system has two modules: the 
source coding module and the channel coding module. The outputs of two correlated sources are first represented 
efficiently into a triple of messages (Wo, Wi, W2) in the source coding module. Then, these messages are reliably 
transmitted over the broadcast channel in the channel coding module. In more detail, the source coding module 
produces three messages Wo, W\ and W 2 , where Wo is a common message to both receivers which contains common 
information about both sources S and T, and W\ and W2 are private messages which contain the individually 
remaining information about the source S and T after extracting the common information, respectively. The 
messages W , W\ and W 2 belong to integer sets Wo = {1, 2, ... , A }, Wi = {1, 2, . . . , Ai} and W 2 = {1, 2, . . . , A 2 }, 
respectively. In general, Wo, W\ and W2 are not independent. Then, the channel coding module wants to reliably 
transmit a message pair (Wo, Wi) to receiver 1 and (Wo, W2) to receiver 2. 

We assume that there is some kind of correlation between two messages W\ and W2, i.e., private messages for 
the receivers can not be chosen independently. 

Definition 9 • The private messages of receivers are said to be correlated, if for every wo £ Wo, there exists 
a set A(wq) such that A(wo) C Wi x W2, and conditioned on Wo = wo the message pairs (Wi, W2) G A(wq) 
are equally likely with probability r^nj^ji , and the message pairs (Wi, W 2 ) ^ A(wo) have probability zero. 

• The private messages of the receivers are said to be independent, if A(wq) = Wi x W 2 for all wo £ Wo- In 
this case, the message pairs (Wi, W2) are equally likely with probability 



W1XW2I ' 



We use bipartite graphs to model the correlation of the messages, i.e., the set A(wo) is taken to be a bipartite graph 
for all wo G Wo- Let us first define a bipartite graph and related mathematical terms before we discuss the main 
problem. 
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Definition 10 • A bipartite graph G is defined as an ordered tuple G = (A\, A 2 , B) where A\ and A2 are two 

non-empty sets of vertexes, and B is a set of edges where every edge of B joins a vertex in A\ to a vertex in 
A 2 , i.e., B C Ax x A 2 . 

• If G is a bipartite graph, let V\{G) and V2{G) denote the first and the second vertex sets of G, respectively, 
and E{G) denote the edge set of G. 

• tfihj) £ E(G), then i and j are said to be adjacent. 

• // each vertex in one set is adjacent to every vertex in the other set, then G is said to be a complete bipartite 
graph. In this case, E{G) = Vi(G) x V 2 (G). 

• The degree of a vertex v e Vi(G) in a graph G, denoted by deg G is the number of edges connected to v 
fori = 1,2. 

• A subgraph of a graph G is a graph whose vertex and edge sets are subsets of those of G. 

Since we consider a specific type of bipartite graphs in our discussion, let us define those bipartite graphs as 
well. Although our main results deal with nearly semi-regular graphs, for the purpose of illustration, we consider 
semi-regular graphs for this section alone. 

Definition 11 A bipartite graph G is called semi-regular \3J$ with parameters (6\, 62, 0[, d^), if it satisfies: 



As an example, two semi-regular bipartite graphs with parameters (4, 6, 2, 3) are shown in Figure Note that 
there exist many semi-regular bipartite graphs with the same set of parameters. 



Definition 12 A bipartite graph G is called nearly semi-regular with parameters (Ai, A2, A' 1; A' 2 , n) for /i > 1 if 



• \V i {G)\=6 i for i=l, 2, 



• Vu G Vi(G), deg G<1 (u) = e' 2 



. VveV 2 (G), deg Ga {v) = e[. 




Figure 4: Examples of semi-regular bipartite graphs with parameters (4, 6, 2, 3) 



it satisfies: 



• \Vi(G)\ = At for i = 1,2, 
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• Vu 6 Vi{G), A^- 1 < deg Gjl (u) < A' 2 n, and e V 2 (G), A'^- 1 < deg G 2 (v) < A' x /x. 
Note that fi is a slack parameter, which determines the range of degrees of vertexes. 



Definition 13 A nearly semi-regular bipartite graph G is said to have parameters (Ao, Ai, A2, A' 1; A' 2 , n) for /i > 1, 
if it satisfies the following conditions: 

• G is the union of Aq disjoint subgraphs G m for m = 1,2, . . . , Ao where G m is nearly semi-regular with 
parameters (Ai, A2, A^, A2, fJ>) as in Definition \12l 

• Vm G {1, 2, . . . , A }, Vi(G m ) = {(m - 1)A, + 1, (m - 1)A 4 + 2, . . . , mAJ /or »=J, 2, 

• K(G0 = U„Li Vi(G m ) for i=l, 2, and E(G) = U*° =1 E(G m ), 

A triple of messages (random variables) (Wo, W\, W%) can be associated with a graph G with parameters (Ao, 
Ai, A 2 , A' x , A' 2 , n) in the following way. Wi G W l and Wj = {1,2, .. . , A,} for i = 0, 1,2, and an edge ((m - 
l)Ai + i, (to — l)Aa + i) £ E(G) denotes a realization of the triple of messages (WojWijWa) = (to, In other 
words, Pr[(W ,W 1 ,W 2 ) = (m,i,j)] = if ((to- 1)Ai +i, (to- 1)A 2 +3) G £(G) and otherwise. So, different 

correlation structures of the messages can be modeled by varying the structure of the graph. Figure illustrates 
an example of a bipartite graph G with parameters (2, 4, 4, 2, 2, 1) composed of two disjoint subgraphs. This graph 
can be associated with a triple of messages (Wo, W\,Wi) where Wj £ Wj for i = 1,2,3 such that Wo = {1,2}, 
and Wi = W2 = {1,2,3,4}. Note that in the figure two subgraphs G\ and G 2 have different edge structures. This 
implies that the correlation of (Wi, W2) depends on Wo. Wo and W\ are independent, and so are Wo and W 2 . 

1 • — — — 

3 — - •3 

- «4 

5 - 

6 • -~C^><C^' *6 

8« ^-- •8 

Receiver 1 Receiver 2 

Messages Messages 

Figure 5: An example of bipartite graph G, with parameters (2, 4, 4, 2, 2, 1), composed of two disjoint subgraphs Gi and G2 
where Wo = {1, 2}, Wi = W2 = {1, 2, 3, 4}. 

3.2 Equivalence classes of graphs 

Let us consider the set of all semi- regular bipartite graphs with fixed parameters (9i,02,O' 1 ,9' 2 ). It is well-known 
that this set can be partitioned into equivalence classes where equivalence relation is permutation and relabeling 
of the vertexes in the graphs. In more detail, one element (or graph) in a class can be obtained from the other 
in the same class by permutation and relabeling of the vertexes. However, if two elements (or graphs) belong to 
different classes, they can not be obtained from each other by permutation and relabeling since they have different 
correlation structures. 
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This means that if we have a transmission system which can reliably transmit the edges of a graph G (i.e., the 
correlation of the message pairs are modeled using G) then this transmission system can be used to reliably transmit 
edges of any graph that belongs to the equivalence class of G. Similarly, if one can construct a source representation 
system that can represent a source pair using a graph G (i.e., the correlation of the index pairs produced by the 
source encoder is modeled using G), then it can be used to represent the source pair using any graph that belongs 
to the equivalence class of G. 

4 Broadcast Channels with Correlated Messages 

In this section we characterize transmissibility of certain correlated messages over a stationary discrete memoryless 
broadcast channel. 

4.1 Summary of Results 



Although, ideally, we would want to use semi-regular graphs for source representation and communication of 
information over broadcast channels, for the sake of analytical tractability, as is typical in Shannon theory, we will 
allow some slack with regard to the degrees of the vertexes of these graphs, and consider the asymptotic case when 
this slack is bounded in some way. 

Definition 14 An (n,r) -transmission system for a nearly semi-regular bipartite graph G with parameters (Ao, Ai, 
A2, A 2; fj.) and a broadcast channel (A", 3^1 5 3^2 , j J/2 1^)) is an ordered tuple (/, 51,32)) consisting of one 
encoding mapping f and two decoding mappings g\ and gi where 



. / : E(G) -» X n , i.e, 

V((m - l)Ai +i, (m - 1)A 2 +j) 6 E(G), assign x n = f(m,i,j) where m £ {1,2, . . . , A } ; i E {1,2, . . ., A x } ; 
andj e {1,2,...,A 2 }, 

• gi ■ W -> Vi(G) for i = 1, 2, i.e., 9l : y? ^{1,2,..., A } x {1, 2, . . . , AJ, 

• such that a performance measure given by the following average probability of error satisfies: 




Encoder 




Figure 6: Transmission of correlated messages over the broadcast channel 



1 




Pr [(g 1 (Y 1 n ),92(Y 2 n ))^((m,i) > (m,j))\X 



n 



f( m ,i,j)] ■ 



(14) 




((m-l)Ai+*,(m-l)A 2 +j)eE(G) 



In Definitional the messages Wq, W\ and W2 are assumed to have the following distribution: 
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• Alphabet of W l is {1,2,..., AJ for i = 0, 1, 2, 

.Pr { ( Wo ^ 1 ^ 2 ) = (m ) M-)} = { if((m-l)A 1+ i,(rn-l)A 2+i)e ^), 

Remark 1 In broadcast channels, the goal of the channel encoder is to reliably transmit two pairs of messages 
(Wo, Wi) and (Wo, W2) to receiver 1 and receiver 2, respectively, over the channel. In terms of graphs, it is to 
reliably transmit edges of a graph which is associated with the triple of messages (Wo, W\, W2) over the channel. 
As an analogy, the conventional Shannon's channel coding theorem in a point-to-point communication scenario can 
be interpreted as finding the maximum number of codewords (colors, if each codeword has a different color) that 
are distinguishable at the noisy receiver. In the conventional broadcast channel with one sender and two receivers, 
the goal is to distinguish colors at the noisy receivers, where the first color, which is common to two receivers, can 
come from one set and the second and the third colors, which are private to each receiver, can come from other two 
sets, respectively, and all possible combination of triples in the three sets are allowed. A natural question to ask is: 
if only a fraction of all possible combination of pairs of colors is permitted depending on the common color, what 
is the maximum size of the sets of these colors which can be reliably distinguished at the receivers. 

Definition 15 A tuple of rates (Rq, Ri, R2, R[, R 2 ) is said to be achievable for a given broadcast channel with 
correlated messages, if for all e > 0, and for sufficiently large n, there exists a bipartite graph G with parameters 
(Ao, Ai, A2, A^, A 2 , fi) and an associated (n,r) -transmission system as defined above satisfying: ^TogAo > Ro — e, 
ilogAj > Ri — e, ilogA^ > R[ — e for i = 1,2, ^log/i < e and the corresponding average probability of error 
t < e. 

Note that in the above definition, we have taken an optimistic point of view. As long as one can find a sequence 
of nearly semi-regular graphs where the number of vertexes and the degrees are increasing exponentially with given 
rates, such that the edges from these graphs are reliably transmitted over the given broadcast channel, we allow the 
corresponding rate tuple to belong to the achievable rate region. The goal is to find the capacity region Cbc which 
is the set of all achievable tuple of rates (Rq, Ri, R2, i? 2 )- In the following we provide an information-theoretic 
characterization of an achievable rate region. This is an inner bound to the capacity region Cbc, and is also a 
per-letter characterization. This is one of the main results of this paper. 

Theorem 1 For a discrete memoryless broadcast channel {X, y2,p{yi, U2\x)), R*bc c ^bc where 
T^*bc = U {(Ro,Ri,R2,R[,R'2) 

p(z,u,v,x) 

i2o<min{I(Z;yi),J(Z;r 2 )}, (15) 

Ri < I{U;Yi\Z), (16) 

R2<I(V:Y 2 \Z), (17) 

R x +R' 2 = R[ +R 2 < I(U; Y X \Z) + I(V; Y 2 \Z) - I(U; V\Z)} (18) 



13 



where Z , U and V are auxiliary random variables with finite alphabets Z, U and V , respectively, andp(z, u, v, x, yi, y 2 ) = 
p(z)p(u,v\z)p(x\z,u,v)p(yi,y2\x) satisfies a Markov chain (Z,U,V) — > X — > (Yi,!^). 

Remark 2 When the private messages are independent, i.e., when all the elements in the set W\ x W2 can occur 
with non-zero and equal probability, the rate region becomes exactly the same as Marton's |12) . However, when the 
private messages are correlated, i.e., only some elements in the set Wi x W2 can occur equally likely, the sum rate 
i?i + i?2 can be larger. As the amount of correlation between the messages increases the achievable rate region also 
becomes larger. 

Remark 3 The limitations of this theorem are as follows. Note that this theorem gives only a partial characteri- 
zation of the set of all nearly semi-regular graphs whose edges can be reliably transmitted over a broadcast channel. 
In the formulation of the achievable rate region, we have the freedom of choosing the correlation of the messages 
for every block-length n. The theorem characterizes the rate of exponential growth (as a function of the number 
of channel uses) of size of certain nearly semi-regular graphs, such that edges coming from any such graph can be 
reliably transmitted over the broadcast channel. This obviously also means that it is possible to transmit edges of 
every graph that belongs to the equivalence class of any of these graphs. However, this fact does not mean that the 
edges of any graph with those parameters can be reliably transmitted. 

4.2 Proof of Theorem [T] 

In this section we prove Theorem ^ by using the method of random coding, random binning, the concept of joint 
typicality of sequence pairs and some concepts from the theory of random graphs [41 j . In addition to using the 
techniques given in |24j . we devise a concept of a " super-bin" , which is a group of consecutive bins, to take into 
account the correlation between the messages. Given a broadcast channel with distribution p(yi, 1/2 1^), consider a 
fixed joint distribution p{z 1 u,v,x) — p(z) p{u,v\z) p(x\z,u, v) where z,u and v are auxiliary random variables on 
Z x U x V. Also, fix e > 0, an integer n > 1, and positive real numbers Rq, and R 2 . 

Random sequences and bin generation: Draw 2 nR ° sequences Z n (m) for m G {1, 2, . . . , 2 nR °}, of length n, 
independently with replacement from A{ n \z) each with probability — ^ where Ai n \Z) is the strongly e-typical 

l-^e (Z)\ 

set with respect to the distribution p(z) which is a marginal of the joint distribution p{z, u, v) and \A\ denotes the 
cardinality of a set A. 

For each m G {1, 2, . . . , 2™*°}, draw 2 n W> Y ^~^ n-length sequences U n (k, m) for k E {1, 2, . . . , 2"( / ( l/ ^il z )- e )} 
independently with replacement from A(U\Z n {m)) with probability \A^j\z n (m)) \ • ^" au ^ ms collection as Ci(m). 
Here, A(U\z n ) is the set of n-sequences u n which are strongly jointly typical with the sequence z n , i.e., for z n € 
Ai n) (Z), the set A(U\z n ) = {u"|(u",z n ) G Ai n) (U,Z)}. Similarly, generate 2™( / ( y ^|z)-e) sequences V n {l,m) for 
/ G {1,2,..., 2™( 7 ( y;Y2 l z )~ e )} independently with replacement from A(V\Z n (m)), where for z n G A^, the set 
A(V\z n ) = {v n \(v n ,z n ) G Ai n) (V,Z)}. Call this collection as C 2 (m). Without loss of generality 2 n{ - I{1J > Yl \ z ^ and 
2n(i(V;Y 2 \z)-(.) are assume( j to be integers. 



14 



Next, for each m in {l,2,...,2 nRo }, define random bins B(i,m) and C(j,m) for i £ {1, 2, . . . , 2 nRl } and 
j G {l,2,...,2 nR2 } such that 

B(i,m) = {U n (k, to) | e [(i - 1) • 2™W i7;Yl l^ _Rl ~ £ ) + 1, i • 2 "( / ( c Wl z )-«i- e )]} (19) 

C(j,m) = {V n (l,m) | I G [(j - 1) • 2 n ( / ( y ^l z )- ife - e ) + 1, j • 2"( J ( vr ^|z)-fi2-e)^ ( 2Q ) 

where without loss of generality 2™( 7 ( c/;Yl l z )~- Rl ~ c ) and 2™( / ( V;Y2 l z )~- R2 ~ e ) are considered to be integers, and [a, b] 
denotes the set of integers from a to b. This imposes the following constraints on i?i and R2: R\ < I(U;Yi\Z) — e, 
and i? 2 < 7(t/;y 2 |^) - e. 



n n 




Figure 7: A subgraph Gi (m = 1) with parameters (2 nRl , 2 nR2 , 2 nR ' 1 , 2 nR * , p). 



Graph generation: As shown in Figure|3 for each m in {1, 2, ... , 2 ni?0 }, a random graph G m can be associated 
with the bins as follows. (1) Vi(G m ) = {(to-1)2" Hi + 1, .. ., m2 nRl } and V 2 (G m ) = {{m-l)2 nR ' 2 + l, m2 nR2 }, (2) 
V(i,j) £ V\ (G m ) x V2 (G m ) , (i,j) G E(G m ) if and only if there exists in B(i 1 m) xC(j, to) at least one e-strongly jointly 
typical sequence pair that belongs to A(U, V\Z n (m)), where A(U, V\z n ) is the set of pairs of sequences that are jointly 
e-typical with the sequence z n , i.e., for z n £ A^ n) (Z), the set A(U,V\z n )= {{u n , v n )\(u n , v n , z n ) £ A^ ] (U, V, Z)}. 
Let G denote the random graph that is the union of 2 nR ° random graphs G m for to G {1,2, ... , 2 nR °}. 

Codebook generation: A random channel codebook C can be generated from the graph G as follows. For 

every to G {1, 2, . . . , 2 nRo }, and every £ E(G m ), first find one pair of sequences (U n (k, to), V n (l, to)) G 

A(U, V\Z n (m)) n (B(i,m) x C(J, to)). Then draw a random codeword X n (m,i 1 j) = f{m,i, j) uniformly from 
A(X\U n (k),V n {l),Z n (m)), where A(X \u n , v n , z n ) = {x n \(x n , u n , v n , z n ) £ Ai n) (X, U, V,Z)}. Thus the size of the 
codebook is equal to the size of the edge set of the graph G, i.e., |C| = |i?(G)|. 

Encoding error events due to the degree condition: Before we proceed to the encoding and decoding 
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procedure, we need to make sure that the generated codebooks satisfy certain properties. If vertexes of the graph 
G do not satisfy the degree conditions, the message pairs can not be transmitted with arbitrarily small probability 
of error. Let 

A = I(U; Y X \Z) + I(V; Y 2 \Z) - I(U; V\Z) - 2e, (21) 

and choose R[ = A — R 2 and R' 2 = A — R\. Since < R[ < Ri for i = 1, 2, we have the following conditions on i?i 
and R 2 : 

max{R 1 ,R 2 } < A< R x + R 2 . (22) 
In summary, the nonnegative tuple (fix, R 2 , R[, R 2 ) that we use for the random coding satisfies: 

Rx<I(U;Yx\Z)-e, R x < I(V;Y 2 \Z) - e, (23) 

Rx + R' 2 = R 2 + R[ = I(U; Y X \Z) + I(V; Y 2 \Z) - I(U; V\Z) - 2e. (24) 

For a precise characterization of the error events, we need a function of e, and in turn certain properties of typical 
sets. For any triple (U, V, Z) of finite- valued random variables, there exists |35) a continuous positive function ei(e) 
(that depends on the triple) such that (a) ei(e) — > as e — > and (b) for all e > (sufficiently small), there exists 
an integer iVo(e) > such that Vn > AT (e) the following conditions hold simultaneously 



for all z n 6 A [ r\z) 



• V(u",* n ) e At\u,z) 



\f(v n ,z n ) e AT\v,Z) 



2 n(H(U\Z)-e 1 ) < |A((7|z")| < 2™W l/ l Z )+^ (25) 

2 n(H(v|z)- ei ) < |>l(V|,z n )| < 2 n{H{y \ z)+ei) (26) 

2 ™(i?(!7,y|2)-e 1 ) < |A([/,7|z n )| < 2"( ff W v 'l z )+ e i) (27) 

2 n(H(V\U,Z)-e 1 ) < z «)| < 2 »Wm*)+«0 ( 28 ) 

2 n(if(C/|V,X)-c 1 ) < ^(C/I^^n)) < 2 n(if(U| V^Rex) . (2 9) 



Coming back to the encoding error event, an error will be declared if either one of the following events occur. 
Let e' > 3ei > 0. 

• Ex: 3i e Vx(G) such that |± logdeg Gil (i) - R' 2 \ > e', 

• E 2 : 3j e V 2 (G) such that |± logdeg G)2 (j) - R[\ > e'. 

Now we show that the probability of these error events can be made arbitrarily small under certain conditions. 
Let us define four events as follows. 
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3i e V\ (G) such that deg< 


3il (i) < 2"^- £ ') 


• £-0,2 : 


3? £ V2(G) such that deg 


S)2 (i) < 2"(«i- e ') 


• ES,i ■ 


3i e V\ (G) such that deg« 


3il (<) > 2™( fl 2+ 6 ') 


• ^0,2 ; 


3? £ V2(G) such that deg 


K>2 (j) > 2"«+^') 



Toward proving the required statements, we need a lemma given in 02] about certain properties of typical sets. 

Lemma 1 For any triple of finite-valued random variables (U, V, Z), any e > (sufficiently small), any two positive 
real numbers R\ and R 2 such that R\ + R 2 > I(U;V\Z), and any z n 6 A[ n \z) consider the following random 
experiment. Generate two collections of sequences Cu{z n ) and Cv{z n ) of size 2 nRl and 2 nR2 from A(U\z n ) and 
A(V\z n ), respectively, with uniform distribution and with replacement. Let P e (z n , Ri, R 2 ) denote the probability 
that \{Cu X C v ) n A(U,V\z n )\ = 0. Then Ve > 0, VR U R 2 withR l +R 2 > I{U;V\Z) andVz n S A^ (Z), 

lim logP e (z n ,Ri,R 2 ) = 00. (30) 

n^oo 77, 

Proof: See Theorem 2.1 of 

The proofs of the next two lemmas use this result. 

Lemma 2 For any e > 0, and sufficiently large n: 

P{EoA<^ for 1 = 1,2 (31) 

Proof. Refer to Appendix 1X1 

Lemma 3 For any e > 0, and sufficiently large n: 

P{Eo ti }<^, fort = 1,2 (32) 

Proof. Refer to Appendix FBI 

Note that £1 = i?o,i Ufi^ and E 2 = -E0.2 U Eq 2 . Thus, according to the Lemma |21 and [31 it is easy to see that, 
for sufficiently large n, P{E X ) < f, and P{E 2 ) < §, if R' 2 = I(U;Yi\Z) - Ri + I(V;Y 2 \Z) - I(U;V\Z) - 2e and 
R[ = I(U; Y t \Z) -R 2 +I(V;Y 2 \Z) -I(U; V\Z) - 2e, respectively. So, it is shown that with high probability we can 
obtain a nearly semi-regular bipartite graph G composed of 2 nRo disjoint subgraphs G m such that for each to, each 
vertex in V\ (G m ) has degree nearly equal to 2 nR ^ and each vertex in V2(G m ) has degree nearly equal to 2 nRl . The 
size of Vi(G m ) is 2 nRl and that of V 2 (G m ) is 2 ,i/?2 . 

Choosing message correlation: If none of the above two error events E\ and E 2 occurs, choose G = G. Clearly 
G has parameters (2 nR ° , 2™ fll , 2 nR2 , 2 ni?1 , 2 ni?2 , 2™ e ). If any of the above two error events occurs, then pick 
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any graph with parameters (2 nRo , 2 nRl , 2 nR \ 2 nR 'i, 2 nR ' 2 , 2™ e ') and call it as G. The distribution of W , W x and 
W 2 is chosen as follows. Pr{(W , W U W 2 ) = (m, = ^ if ((m - l)2 nRl +i,(m- l)2 nR * + j) £ E{G) and 
Pr{(Wo, Wi, W2) — (m, — else. For this graph G, and the given broadcast channel, using the above random 
codebook C, we construct an (n, retransmission system, where r will be specified in the sequel. 

Encoding: Sender transmits the codeword X n (m,i, j) over the channel to deliver two pair of messages (m,i) and 
(m,j) to receiver 1 and receiver 2, respectively. 

Decoding: Both Receiver 1 and Receiver 2 first find the unique index rh such that Z n (rh) is jointly typical with 
received sequence Y™ and Y 2 n , respectively. Then, Receiver 1 finds the unique index k such that U n (k,rh) is jointly 
typical with the received sequence Y™ and Z n (m), i.e., {U n (jt,m),Y?,Z n {m)) G d?\u,Y u Z). Similarly, Receiver 
2 finds the unique index 1 such that {V n {l, m),Y 2 n , Z n {m)) G A ( / l) (V, Y 2 , Z). Then, each receiver finds the decoded 
private messages i and j such that U n (k,m) G B(i, m) and V n (l,m) G C(j,rh), respectively. Otherwise, an error 
will be declared. 

Probability of error analysis: So, the probability of error P(E) can be given by 

P(E) = P{E X U E 2 )P(E\E 1 U E 2 ) + P{E n E{ n E c 2 ) (33) 
< P(Ei U E 2 ) + P{E n El n El) (34) 

The second probability in the above equation can be bounded as given in the following lemma. 

Lemma 4 For any e > 0, and sufficiently large n, 

p(e n El n El) < I (35) 

provided 

i? <min{/(Z;y 1 ),/(Z;y 2 )}-e. (36) 

(37) 

Proof. Refer to Appendix IO 

Therefore, by applying the union bound we have P(E) < P{E X ) + P(E 2 ) + P(E n Ef n Ef) < e. 

Since in every realization of random codebooks, we have chosen a nearly semi- regular graph G with parameters 
(2 n -R°, 2» 1 -Ri j 2 nR2 , 2 nRl , 2 nR - 2 , 2 ne ), and averaged over the ensemble of random codebooks, the average probability 
of error is smaller than e, there must exist a graph G with parameters (2 nR ° , 2 nRl , 2 nR2 , 2 nRl , 2 ni?2 , 2 ne ) and a 
codebook such that the average probability of error is smaller than e. This is true only under the condition given 
by the statement of the theorem. Hence, the proof of Theorem ^ has been completed. ■ 
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4.3 The Capacity Region of a Broadcast Channel with One Deterministic Component 
with Correlated Messages 

In this section, we consider the capacity region of broadcast channels with one deterministic component with 
correlated messages, CbCi when there is no common message, i.e., when Rq — 0. In this case, as expected, we can 
provide a converse coding theorem. 

Theorem 2 For a discrete memoryless semi-deterministic broadcast channel where y\ = fi(x) and the probability 
of y 2 given x is p(y 2 \x), TZ* BC = C B c where 

K*bc = U {(-Rij #2, R[, R' 2 ) 

Ri < H(Yi) (38) 

R 2 <I{V;Y 2 ) (39) 

R x +R' 2 = R[ +R 2 < H(Yi\V) + J(V; Y 2 )} (40) 

for some p(v,x) onVxX, where V is an auxiliary random variable with finite alphabet V such that V — > X — > 
(Yi,Y 2 ), \V\ < \X\+2. 

Proof: See Appendix iDl 

5 Representation of Correlated Sources into Graphs 

In this section, we consider the problem of representation of correlated sources into graphs for transmission over 
broadcast channels, i.e., source coding module in broadcast channels with correlated sources. This problem can be 
interpreted as Gray-Wyner problem with correlated messages. In in this setup, the source encoder can access both 
sources to be transmitted, but the source decoders can not collaborate with each other. We consider two correlated 
sources (S,T,p(s,i)). 

5.1 Summary of Results 

In this problem, the goal is to reliably represent two correlated sources into a triple of messages (Wo, W\, W 2 ) which 
can be associated with a nearly semi- regular bipartite graph G with parameters (Ao, Ai, A2, A' 1; A' 2 , /j,) as defined 
in Definition EH As shown in Figure |H1 the output of source encoder is the triple Wo, W\ and W 2 . We assume that 
two pairs of messages (Wo, Wi) and (Wo, W 2 ) are sent to receiver 1 and receiver 2, respectively, over the channel 
without error. From the received message pairs, the two source decoders wish to reliably reconstruct the original 
source sequences S n and T™, respectively without communicating with each other. For ease of exposition let is 
consider two simple functions 1\ and I 2 both having domain as Z x Z and range as Z given by Ii(a,b) = a and 
I 2 (a, b)=b for all (a, b) G Z x Z. 
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Figure 8: Gray-Wyner problem with correlated messages 

Definition 16 An (n,r) -transmission system for a nearly semi-regular bipartite graph G with parameters (Ao, 
Ai, A 2 , A' 1; A' 2 , n) and a pair of correlated sources (S, 7~, p(s, t)) is an ordered tuple (f,gi,g 2 ), consisting of one 
encoding mapping f and two decoding mappings g\ and g 2 where 

• f :S n x T" -> E(G), 

. g 1 :V 1 (G)^S n ,g 2 :V 2 (G)^T n , 

• such that a performance measure given by the probability of error satisfies: 

r = Pr{( gi (h(f(S n , r™))), g 2 (I 2 (f(S n , T")))) ± (S n ,T n )}. (41) 

Now we define achievable rates for this problem as follows. 

Definition 17 A tuple of rates (Ro, R\, R 2 , R\, R' 2 ) is said to be achievable for a pair of correlated sources (S, T ,p{s, t)) 
(for transmission over broadcast channels), if for all e > 0, and for all sufficiently large n, there exists a bipartite 
graph G with parameters (A , Ai, A 2 , A' 1; A' 2 , /i) and an associated (n,r) -transmission system as defined above sat- 
isfying: i log A, < Ri + e for i = 0, 1, 2, ^ log A^- < R'j +e for j = 1,2, i log /i < e and the corresponding probability 
of error r < e. 

The goal is to find the achievable rate region IZgw which is the set of all achievable tuple of rates (Ro , R\ , R 2 , R[ , R' 2 ) . 
We have obtained an inner bound to the achievable rate region. It is one of the main results of this paper and is 
given by the following theorem. 

Theorem 3 C IZgw where 

T^-hw — U {(Ro,Ri,R2 1 R[,R 2 ) 

p(z\s,t) 

Ro>I(S,T;Z), 



Ri > H(S\Z), R 2 > H{T\Z), 
R[>H(S\T,Z), R' 2 >H(T\S,Z), 

where Z is an auxiliary random variable with a finite alphabet Z such that p(z, s,t) — p(s,t)p(z\s,t). 



(42) 
(43) 
(44) 



Remark 4 Note that we can obtain Rq + R\ + R' 2 = Rq + R[ + R 2 > H(S,T) by combining the conditions in 
Theorem^ This means that in every nearly semi-regular bipartite graph used to represent the source, as expected, 
the total number of edges must be greater than or equal to 2 nH ( S ' T ) . 
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5.2 Proof of Theorem [3] 

In this section, we present a proof of Theorem [3| We use a random coding procedure and the notion of strongly 
jointly typical sequences. Let us consider a fixed finite set iJ, and a joint probability distribution p(z, s, t) on 
Z xS x T. Also fix e > 0, and real numbers Ro, Ri, R2, R' i = Ri~ I(S\ T\Z) for i — 1,2. Without loss of generality 
we assume that R4 > I(S; T\Z) for i = 1,2. 

Random sequence generation: First, draw 2 nR ° sequences Z n (m), m G {1,2, ...,2 nRo }, of length n, in- 
dependently from the strongly e-typical set A { t n) (Z). That is, P{Z n m = z n } = * if z n G A ( e n) (Z), and 
P{Z™ = z 11 } = if z n <£ A { r ] {Z). For every m G {1, 2, . . . , 2 nito }, draw 2 nitl sequence from ^ n) (5|^ ,1 (m)) inde- 
pendently, equally likely and with replacement. Call this collection B(m). Denote the zth sequence as S n (i,m). By 
collecting these 2 nR ° bins the first codebook Ci can be obtained. In other words, Ci = {B(l),B(2), . . .,B(2 nR °)}. 

Similarly, for every me {1, 2, ... , 2"^°}, construct a bin C(m) containing 2 nR2 sequences from A* (T\Z n (m)). 
Let the jth sequence in C(m) be denoted by T n (j,m). The second codebook C2 can be similarly generated, i.e., 
C 2 = {C(1),C(2),...,C*(2"^)}. 

Graph generation: Now we generate a bipartite graph G from the codebooks Ci and C2 as follows: 

• G is composed of 2 nRo disjoint subgraphs G m for m G {1,2,..., 2 nRo }, 

• F i (G) = U 2 ;"^(G m ) for i = 1,2, E(G) = U^E(G m ), 

• Vme{l,2,...,2"' Ro }, Fi(G m ) = {(m- l)2 nRl - 1, . . . ,m2 nRl }, V 2 (G m )={(m - 1)2"* - 1, . . . ,m2 nR2 }, 

• VmG{l,2,...,2 nRo }, G E(G m ) if and only if (S n (i,m), T"(j,m), Z"(m)) G Ai n) (5,T,Z). 

Encoding error events: Before we proceed further, let us make sure that the generated codebooks satisfy certain 
properties. If the vertexes of G do not satisfy certain degree requirements, we may not be able to reliably represent 
the sources using this graph. For the triple (S,T,Z), consider the function ei(e) as defined in (|25|I - H29|I . Let 
e' > 3ei > 0. An encoding error will be declared if either one of the following events occurs. 
Ei : 3i G Vi(G) such that |± logdeg^ ~ R' 2 \ > e', 
E 2 : 3j G V 2 (<G) such that |ilogdeg G 2 - R[\ > e'. 

Using the following lemma it can be shown that the probability of these two events can be made arbitrarily 
small for large n. 

Lemma 5 For any e > 0, and sufficiently large n, we have P{Ei) < |, for i = 1, 2. 

Proof: The proofs of these two results are similar, respectively, to those of Lemma |5| and Lemma |21 Hence for 
conciseness we omit the proof. 

So by LemmaEl P{E{) + P(E 2 ) < |. Hence we can obtain a bipartite graph G where each vertex in Vi(G) has 
degree nearly equal to 2 ni?2 and each vertex in V 2 (G) has degree nearly equal to 2 nRl . 
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Choosing message correlation: If none of the above error events E\ and E<x occurs, then choose G = G. If any of 

the above two error events occurs, then pick any graph with parameters (2 nRa , 2 nRl , 2 nR2 , 2 nRl , 2 nR ? , 2 ne ) and call 
it as G and no guarantee will be given regarding the probability of decoding error. For this graph G, and the given 
correlated sources S and T, using the above random codebooks Ci and <C 2l we construct an (n, retransmission 
system, where r will be specified in the sequel. 

Encoding: If (E 1 U E 2 ) occurs, then the encoder is some arbitrary mapping S n x T n — > E(G). Otherwise, for a 
given (S n ,T n )eAi n) (S,T), find an index to, for to S {1,2, ...,2 nRo }, such that (S n , T n , Z n {m)) e A { "\S,T,Z). 
If there is no such index m, let m be a random index chosen uniformly from {1, 2, . . . , 2 ni?0 }. Also, find a pair of 
indexes (i, j) where i (and j, respectively) is the index such that S n (i, to) = S n (and T n (j, to) = T n , respectively). 
If there is no such sequence pair let (i,j) be a random edge from the corresponding sub-graph of G. The encoder 
sends (m,i) and (m,j) to receiver 1 and receiver 2, respectively. 

Decoding: Given the received index pair (m,i), receiver 1 declares S n — S n (z, to) . Similarly, given (m,j), receiver 
2 declares f n = T n (j,m). 

Probability of error analysis: Let E denote the event {(<?i(m, i), g2(m, j)) ^ (S n , T n )}, that the reconstruction 
vector pair is not equal to the source vector pair. The probability of error P(E) can be given by 

P{E) = P(E 1 UE2)P{E\E 1 UE 2 )+P(E C\ El D E%) (45) 
<P(E 1 UE 2 )+P(Er\E1r]E^. (46) 

The second probability in the above equation can be bounded as given in the following lemma. 

Lemma 6 For any e > 0, and sufficiently large n, 

P(E n El n E c 2 ) < ^ (47) 

provided 

R > I(S,T;Z) + e 2 (e), R\ > H(S\Z) + ei(e), R 2 > H{T\Z) + e x (e), (48) 
where e 2 (e) — » as e —> and 62(e) > 0. 

Proof: Refer to Appendix El 

Thus, P{E) < P(Ei) + P{E 2 ) + P{E n J5f n ££) < e. Therefore P(S) < e for sufficiently large n and under the 
conditions given by the theorem. In every realization of random codebooks we have obtained a graph G with the 
same set of parameters, and averaged over this ensemble, we have made sure that the probability of error is within 
the tolerance level of e. Hence, the proof of the direct coding theorem is completed. ■ 

5.3 Outer bound 

In this section we provide a partial converse for Theorem [3] 
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Theorem 4 K G w C 72.; 



GW 



where 



■GW — 



[J {(Ro, Ri, R2, R[, R' 2 ) ■ 



p(w \s,t) 



Ro>I(S,T;Z), 



(49) 



i?! > i? 2 > H{T\Z), 



(50) 



R 1 +R' 2 = R[ + R 2 > T|Z)} 



(51) 



where Z is an auxiliary random variable such that p(z, s,t) = p(s,t)p(z\s,t). 
Proof: Refer to Appendix iFl 

5.4 An interpretation of the rate region 

We have shown that for sufficiently large block length n, a pair correlated sources (S, T) can be represented into a 
nearly semi- regular graph G with parameters (2 nR °, 2 nRl , 2 nR2 , 2 nR i, 2 nR z, 2 ne ) as shown in Figure Note that 



Figure 9: Graph G with parameters (2 nR ° , 2 nRl , 2 nR2 , 2 nR * , 2 nR ' 2 , 2 nE ') composed of 2 nR ° subgraphs G m with parameters 



many different graphs can be used to represent a pair correlated sources without increasing the amount of "global" 
redundancy i.e., the sum rate satisfies: Rq + Ri + R' 2 = H(S,T). This can be seen from the fact that the total 
number of edges in the graph satisfies: \E(G)\ « 2 n ( I ( s - T '' Z * }+H{ - s - T \ z ^ — 2 nH ( s < T \ Further, the rate of the common 
message can vary from to H (S, T). As examples, let us consider some special cases as follows. 

• Case A: If I(S,T;Z) = 0, then R = 0, Ri = H(S), R 2 = H(T), R[ = H(S\T), R' 2 = H{T\S). Roughly 
speaking, this corresponds to the typicality-graph of (S,T). 

• Case B: If I(S, T; Z) = H(S, T), then R = H(S, T), R 1 = 0, R 2 = 0, R[ = 0, R' 2 = 0. 

In particular, consider the case when Ro = C(S;T), where C(S;T) denotes the common information of Wyner 
[SSj. In this case, Ro = C(S;T), R x = H{S\Z), R 2 = H(T\Z), R[ = H(S\Z), R' 2 = H{T\Z) since S and T are 
conditionally independent given Z. So, each subgraph G m becomes nearly complete. This also means that the 
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private messages for receiver 1 and 2 become nearly independent. Note that a subgraph G m can not be complete 
if R < C{S; T) because, by the definition, C(S; T) is the infimum of I(S, T; Z) such that S -> Z -> T. 

In summary, Case A can be thought of as situated at one end of the spectrum, and Case B as situated on the 
other end of the spectrum. For every value of I(S,T; Z), we get an equally efficient representation of the sources 
into a nearly semi-regular graph. Here, an efficient representation means that the cardinality of the edge set of a 
graph used for representation (of n source samples) is nearly equal to 2 nH ( S ' T \ 

6 Interpretation of the Example in Section 12.41 

In this section, we revisit the example of Section 12.41 We analyze and interpret this example from the perspective 
of graphs and the proposed coding scheme. This example deals with transmission of two correlated sources over a 
deterministic broadcast channel. This example can be considered as a special case where the typicality-graph of the 
sources and that of channel outputs can match exactly. So, we can apply our coding theorem to this case. 

Consider the pair of correlated sources of Section T2.4I S = T = {0,1} with distribution p(0,0) = p(0, 1) = 
p(l, 1) = 1/3, and the Blackwell channel with X = {1, 2, 3}, and ^ = {0, 1} for i = 1, 2 with conditional distribution 
p(0,0|l) = p(0,l|2) = p(l,l|3) = 1. Let a = fa (1/3) and (3 = log 3- fa (1/3). It follows that H(S) = H(T) = a 
and H(S\T) = H(T\S) = (3. Consider the rate tuple (0, a, a, (3, (3). Clearly this point belongs to TZq w as this 
tuple corresponds to the typicality-graph of (S, T,p(s, t)). Note that the above broadcast channel is a deterministic 
broadcast channel. Using the arguments presented in Section 14.31 it can be shown that for any deterministic 
broadcast channel, for the special case when Rq = 0, the capacity region Cbc is given by 

\J {(R!, R 2 , R[, R' 2 ) : Ri<H(Yi) for i=l,2, Ri + R' 2 = R 2 + R[ < H(Y U Y 2 )}. (52) 

pO) 

Choosing X equally likely over {1,2,3}, it follows that the rate tuple (a, a, (3,(3) belongs to Cbc, which, of 
course, implies that (0, a, a, (3, (3) belongs to Cbc- In this case the distribution of the channel output (Yi, Y 2 ) is the 
same as the distribution of the sources (S, T). Further, the graph corresponding to the tuple (0, a, a, (3, (3) in source 
coding is the same as the graph corresponding to the tuple (0, a, a, (3, (3) in channel coding. 

In summary: (1) there is a nonempty intersection between the achievable rate regions of the source coding 
module and the channel coding module, and (2) the graph G associated with the source coding module matches (is 
identical) with the graph G of channel coding module. Hence the given sources (S, T) can be reliably sent over the 
given broadcast channel. 

7 Conclusion 

We have considered the problem of transmission of correlated sources over broadcast channels. We have considered 
a graph-based modular architecture involving two components: a channel coding component and a source coding 
component. The graphs are used to model the correlation between the messages. Correlated sources are first mapped 
into such graphs, and the edges coming from these graphs are reliably transmitted over a broadcast channel. 



24 



We have given a partial characterization of the set of all graphs that can be used to represent a given pair of 
correlated sources, and similarly given a partial characterization of the set of all graphs such that edges coming 
those graphs are reliably transmitted over a given broadcast channel. We have also considered special cases such as 
deterministic broadcast channels and broadcast channels with one deterministic component, where converse results 
are provided. We have applied this analysis to the case of transmission of a triangular source over a Blackwell 
channel as an example. The goal of this work is to show that graphs may be used as discrete interface in this 
modular approach to multiterminal communication problems. 

Appendix 

A Proof of Lemma [2] 

The event £0,1 can be considered as 

£ o,i = (J U E oAhm) (53) 

i — 1 m— 1 

where E ,i(i, m) is the event that deg 6|1 ((m - l)2 nRl + i) < 2 n ^~ e '\ 

Note that each vertex in Vi(G) is only connected with a subset of V^Gm) for some to, m £ {1,2,..., 2™^°}. 
We define super-bins B(p, m) and C(q, m) for p 6 [l,2 n ( R i~ e )] and q 6 [1 ; 2™(- R 2 _e ~>], each of which is a union of 
2n(Ri+R 2 -A+e ) consecu ti V e bins B(i,m) and C(j,m), respectively, i.e., 

p2n (R 1 +R 2 -A + e') 

B(p,m)= (J B(i,m), (54) 

i=(p-l)2"<«i+ ii 2-^+«')+l 

C(q,m) = U C(j,m) (55) 

i=( 9 -l)2"(«i+«2-A+«')+l 

The size of each super-bin B(p,m) and C(q,m) is 2 n< - I< - u - Yl \ z '>- e - R 'i +e "^ and 2 n ^n v ^\ z )~ e - R, 2+^\ respectively. 
Recall that R t + R' 2 = R 2 + R[ = A. 

Before we proceed further, let us observe that B(i, to), B(p, m) and C\(m) are collections of 2™( / ( f/;Yl l z )~ e ~- Rl ), 
2n(i(U;Y 1 \z)-e-B, 1 +e ) anc j 2 ra ( 7 ( £/ ; y i I z ) - £ ) ; respectively, of random sequences. Then, the event Eo } i(i,m) can be 
expressed as 

2»(Hi-«'> 

£ , o,i(«,m)c |J E ,i(i,q,m) (56) 
9=1 

where Bo,i(*j 9j to ) is the event that (-B(i, m) x C(g, to)) (~l -A(f, V\Z n (m)) is empty. So, by using the union bound, 
the probability of this event P(-Eo,i(«, to)) can be bounded as: 

|J ^,i(*,g,m)j (57) 
< P{Eo,i{i,q,m)) (58) 

q=l 
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^ E E 77(i^ P(Eo,i{i,q,m)\Z»(m) = f) (59) 
Therefore, for sufficiently large n, the probability of the event i?o,i can be bounded by applying the union bound: 



2" R 1 2" R 

P(E ,i) < E P(EoA^m)) (60) 

i= 1 m— 1 

<EE E E -(^y— W,i(^,m)|Z™(m) = z«) (61) 

i=l m =l g=l ,„ 6A C») W 1^ l Z JI 

(a) , 

%' (63) 



where (a) is from Lemmas because 

1 



log | J B(z,7TT.)||C(g, m)| = I(U;V\Z) + e' , (64) 



and M > is a sufficiently large number satisfying M > R + Ri + R' 2 — e' . 

In a similar way, we can also show that P(Eq ^) < ^ for sufficiently large n. I 

B Proof of Lemma [HI 

The event Eq 1 can be expressed as 

E o,i = U U £ o,i(»^) (65) 

i—l m— 1 

where ^ m) is the event that deg G1 ((m - l)2 nRl + i) > 2 n ( R '?+ e '\ 
Let V(i,m) = {j : \A(U,V\Z n (m)) n (£(«,m) x C{j,m))\ + 0}. Then, 

whe« * M .{ J' « WV|r(.))n(«(i.)»«w m 

3=1 L ' 

In particular, 

PWj) = 1} = P{|A(Z7, F|Z n (m)) n (fl(i,m) x C(j,m))| ^ 0} (67) 
= E | J) ( P{\A(U,V\Z n (m)) n (B(i,m) x C(j,m))\ ± 0\Z n (m) = z"} (68) 

2n(A-R 1 -R 2 +3e 1 ) f^Q\ 



< 1 2 w ^^i y il^- J? -i-^2 n(J(V;Y " 2|z) ~- R2 ~ £) 2~" (/([7;y|z) ~ 3ei) (69) 



where (a) is obtained by applying the union bound, and from the property of strongly jointly e- typical sequences |33| : 
for all z" G Ae (2'), for a randomly and independently chosen U n G A(J7|z") and V n G A(V|z"), for sufficiently 
large n, the probability that (U n , V n ) G A(J7, V\z n ) is bounded by 

2 -n(/(C/,V|Z)+3e 1 ) < p{(f7« 7 V™) G A(E/, < 2-™( / ( C/ > y l Z )- 3ei ) (71) 
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where we have used (gg) , and (|27|) . 

So, the expectation of |D(i, m)| can be bounded as follows. 



Now, 



E\V(i,m)\ = ^P{V(i) = l} 
3=1 

< 2™(' R 2+ 3e l) 

P{f?5 il (i > m)} = P{|X>(*,m)| > 2"(^-Bi+ e ') # } 
< e-° t S{e*l 2J l} 



(72) 

(73) 
(74) 



(75) 
(76) 



for any t > where (b) follows from the Chcrnoff bound 

Now we calculate an upper bound for E{e^ v ^}. Consider an arbitrary sequence z n G A ( - 1 \Z). Let u n [l] be 
the i-th sequence (using some ordering) in A(U\z n ). Recalling the definition given in 1)19(1 and 120(1 . let us denote 
\B(i,m)\ = j3, and if. = (i — 1)(3 + k for k 6 {1,2,..., /?} for notational simplicity. Now consider the following 
sequence of arguments: 



2"B2 

E{e t \ v ^\\Z n {m) = z n } = e! J[ 

3=1 



Z n (m) = z r 



(77) 



|A(I/|* n )| \A(U\z n )\ \A{U\z n )\ 

J2p{U n (i 1 ,m)=u n {h}} £ p{U n (i2,m)=u n [h}}... £ p{U n (ip, to) = u n [l }} 



3=1 



>n 2 



e\ n 



Mil) 



3=1 



C7"(i e , m) = u n [i e ], for (9 = 1, 2, . . . , (3 , and Z n (m) = z n \ (78) 



1 



|A(t/|z")| \A(U\z")\ \A(U\z n )\ 

E E ■■■ E .i ( ri-") ; 

Ji=l i a =l Jfl=l 1 V 1 ;l 

2«R2 



* n 



3=1 



U n (i e ,m) = u n [l g ], for (9 = 1,2,. ..,/3 , and Z"(m) = z" ^ (79) 



\A(U\z n )\ \A{U\z n )\ \A(U\z")\ 

E E ■■■ E 



; 1= i z 2 =i 

2 „R 2 



2,8=1 



\A{u\z n )\e 



3 = 1 



,*^(3) 



U n (i e ,m) = u n [l g ], for (9 = 1, 2, . . . ,0 , and Z n (ro) = z n \ (80) 



where (c) is from the fact that ipU) ,s are independent when the outcomes of U n {i±, to), U n {i2, to), . . . , U n (iff, to) 
and Z n (m) are fixed. Let us denote 



Pj = P{ip{j) = l| U n (ig,m) = u n [Z e ], for = 1,2, ...,/? , and Z n (m) = z"}. 



(81) 
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Then, 



E{e*+W\ U n {i g ,m) = u n [l ], for 9 = 1, 2, . . . , (3 , and Z n (m) = z n } = e% + 1 • (l~Pj) (82) 

= l-p i (l-e t ) (83) 
< g-PiCi-e 4 ) since i _ a; < e -*. ( 8 4) 



So, 



,« 2 



J'=l 


U n (i e ,m) =u n [l e ], for 6> = 1,2,. 


..,j8 , and Z"(m) = z"} 


(85) 


2 nH 2 

< n eK(et " 


-i) 




(86) 


= exp|(e* - 






(87) 


= exp|(e* - 


l)^{|2?(»,m)|| C/ n (i e ,m) = u n [I« 


], for = 1,2,...,/? , and Z n {m) = z n ) j 


(88) 



Then, 



\A(U\z n )\ \A(U\z") 



J\T>(i,m) 



\Z n {m)=z n }< £\"E 



|A(C/|z»)l /3 



z 1= i ^=1 

expj(e* - l)^{|D(i,m)||Z7 n (i e ,m) = w n [Z e ], for 9 = 1, 2, . . . ,0 , and Z n (m) = z n ) 

(89) 



(d) 

< expi (e 1 - 1)2 



n(R' 2 +3ei ) 



(90) 



where (d) is obtained because £7 { \V(i, m)\ | U n (ig,m) — u n [lg], for = 1, 2, . . . ,0 , and Z n (m) — z™} is bounded 
by 2™( fl 2+3ei) ( t he same as the bound on the unconditional expectation as in inequality §74$ ) regardless of the 
particular sequence collection u n [Zi], u™^], ■ ■ ■ ,u n [l(j] and z n . To see this crucial step clearly let us rewrite pj (see 
Q81Jlh using the union bound on the outcome of the bin C(j,m), as follows: 



Pi 



< 2n(I(V;Y 2 \Z)-R 2 -e) p 



P 
8 = 1 



(91) 



where Fg is the event that a sequence drawn from A(V|.£™) belongs to A(V"|i4 n [Zg], z n ). Since all sequences in B(i, m), 
given that Z n (m) = z n , are drawn from A(U\z n ), we have using (|26|l and l|28() 



<- 2 n(^(V r ;r 2 |Z)-ii 2 -e)2n(-f((7;ii|2)-fl 1 -e)2-n(/(i/;V|Z)-3e 1 ) _ 2"(A-fl 1 -ii 2 +3ei) 

Therefore, for f > 0, 

pn(fl 2 +3ei) 



i^JS^ftm)} < e-^exp^e* - 1)2"^ 
= exp^ - at + (e* - 1) 2 



n(ii 2 +3ei) 



= b 



(92) 

(93) 
(94) 
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To get a tighter upper bound, let us denote f(t) = —at + b(e t — 1), for t > 0. Then, f'(t) = —a + be* and 
f"(i) = be* > 0. So, /(£) has the minimum value when t = In (|J . 
Thus, P{£Jq,i(*! m )} is bounded as 

P{£o a (i,m)} < expj -aln(~) + a-bj (95) 

where a = 2"( 7? 2+<0 and b = 2 n( - R 2+ 3 ^ . 
So, 

P{P *,i(*, to)} < exp |„2"(^ +3ei )[2"( e '- 3ei ) ln(2"( £ '~ 3ei )) - 2 ni - fJ + 1]} (96) 
= exp{ - 2™^ +3ei ^} (97) 
<exp{-2"^+ 3ei )} (98) 

where r\ = 2 n ^~ 3 ^ ln(2"( £ '- 3ei )) - 2 n< - £ '- 3e ^ + 1. Note that 77 > since (i) e' > 3ei and (ii) for x > 1, g(x) = 
xln(a;) — x + 1 is increasing function of x and g(x) > 0. Further g{e) = 1. 
Therefore, for sufficiently large n, by applying the union bound, 

P{Ki} = P{ U U K.ifr™)} (99) 

i—1 rn — 1 
i—1 rn—1 

< 2 n(Ro+Rl) exp{ - 2™ ( ^ +3ei) } (101) 
= exp{n(P + Pi) In 2 - 2 n{R ^ +Sei) } (102) 

< ^ (103) 

where (e) is from the fact that n(Ro + Pi) In 2 is linearly increasing but 2™(- R 2+ 3ei ) i s exponentially increasing as n 
increases. 

In a similar way, we can also show that P{Po 2} < T2 l° r sufficiently large n. ■ 

C Proof of Lemma HI 

Now let us calculate the probability P(E H EfD E^)- Without loss of generality, let us assume that the outcome of 
the message triple is given by: (Wo,Wi,W2) = {m,i,j). Let (U n (ki,m),V n (lj,m)) be the pair of sequences that 
are jointly typical in the bin B(i, to) and C(j, to). Consider the following error events. 

E 3 : (Z n (to) , U n (h , to) , V n (l 3 , m), X n , Y?, Y?) $ (Z, U, V, X, Y U Y 2 ) 

E4. Decoding step fails at receiver 1, i.e., 3 to ^ to such that (Z n (m), 1^™) G Ae(Z,Y±) or 3 k ^ ki such that 
(U n (k 1 m),Y 1 n ,Z n (m)) G 4 n) (17, Yi , Z) 

E5: Decoding step fails at receiver 2, i.e., 3 to ^ to such that (Z n (fh),Y 2 n ) £ Ac (Z, Y2) or 3 I ^ lj such that 
(V n (l,m),Y 2 n ,Z n (m)) G Ai n) (V,Y 2 ,Z) 
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It is easy to see that, for sufficiently large n, P{E%) < using the Markov lemma gHH]; P ( E *) < T' if < 
I(Z; Yi)-e and P(E 5 ) < f , if H < I(Z;Y 2 )-e. Recall that for all m G {1, 2, . . . , 2 nR °}, |Ci(m)| = 2 n( ^^ u ^ z ^ 
and \C 2 (m)\ = 2 n ^ v ^\ z )-^ . 

So, 



p(e n n £ 2 C ) = P(u 5 i=3 E t n n £ 2 C ) (104) 
<£p (An (105) 

i=3 

Hence, the proof of Lemma 01 has been completed. ■ 

D Proof of Theorem [2] 

Here we provide the converse part, as the direct part follows from Theorem^ The proof is very similar to Marton's 
proof |12| of the outer bound of the capacity region of semi-deterministic broadcast channels. Consider any sequence 
(indexed by n) of (n, r(n)) transmission systems for a sequence of bipartite graphs G„, respectively, with parameters 
(1, 2 nRl , 2 nR2 , 2 ni? i , 2 nR i , fi(n)) and the broadcast channel with one deterministic component such that r(n) — * 0, 
and i log^(n) — > as n — > oo. 
By Fano's Inequality, 

H(Wi,Wa|li , ,iy) < T{n)-n{Rx+R 2 )+H{ T {n)) < ne„ (106) 
H{Wi\Y?) < t{ji) ■ nRi + H(t(ji)) < ne n (107) 
H(W 2 \Y 2 n ) < r(n) ■ nR 2 + H (r(n)) < ne„ (108) 

where e„ — > as r(n) — > 0. We can bound the rate i?i as 

nRi = H(Wi) = I{Wx;Y{ l ) + H(Wx\Y?) (109) 
< I(Wi;Y?) + ne n < H(Y{ 1 ) + ne n (110) 

n 

<^H{Y u ) + nt n (111) 

i=l 
1 " 

^i?l<-V^)+£n. (112) 

n 



i=i 



i?o can be bounded as 



nR 2 = H{W 2 ) = /(W 2 ; Y 2 ") + {W 2 \Y 2 n ) (113) 

n 

< I(W 2 ;1T) +ne n = ^IiWiiYnlYp-V) + ne n (114) 



i=l 



< Y} H ( Y 2i) - H{Y 2i \W 2 ,Yt\Y?{ i+ i))]+ne n (115) 

i=l 

n 

= ]T 7(V i; y 2i ) + ne„, by defining V t = {W 2 , Yj-\ Y$ i+1) ) (116) 



i=l 
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1 ™ 

=>R 2 <-J2l(Vi;Y 2i ) + e n . (117) 



n 
1=1 



Using the fact that (a) the channel is discrete memoryless and is used without feedback, and (b) Y\ is a 
deterministic function of the channel input, it can be easily shown that Vi — » X; — > (Yu,Y 2 i). Before we bound the 
sum of rates R\ + R' 2 , let us recall the following identity from 03]: for any two sequence of random variables A n 
and B n , 

n n 

Y / nA^ 1) ;B l \B^ +1) ) = Y / nB^ +1 y,A\A^). (118) 

i=l i=l 

The number of all possible pairs of messages (WijWb) which have non-zero probability is lower bounded by 
2 n(fl 1 +^) jU -i = 2n(Rl+R 2 ) M -i_ SO; we can bound the sum rate n(Ri + R' 2 ) as 

n{R 1 +R' 2 )<H(W 1 ,W 2 )+ log (119) 
= H{W 2 ) + H (W 1 \W 2 )+ log n (120) 
= I(W 2 ;Y 2 n ) + H{W 2 \Y 2 n ) +I{W 1 ;Y 1 n \W 2 ) + H{W X \Y?,W 2 \ + log/i (121) 

<nc„ <H(Wx\Y; l )< n£" 

< iTlWa) + I(W 2 ;Y 2 n ) + 2ne n + log/i (122) 

= ^(y/ 1 ) - I(W 2 :Y 1 n ) + I(W 2 ;Y 2 n ) + 2ne» + log/i (123) 



(a) 



£(/(W 2 , n<) - I(Y{l i+l) ;Y u )) 



^/(VK 2 ,y 2 (j 1) ;Y 2 + 2ne„ + log/i (124) 



( = } £ ffO^i) + £ [7(y i; y 2 i) - Ku)] + 2ne„ + log/i (125) 

z=i i=i 

n 

= [H{Yii\Vi) + I(Vi;Y 2i )] + 2ne„ + log/i (126) 

i=l 

1 - 1 

^Ri+R 2 <-J2(H(Y li \V i )+I{V i ;Y 2i )) + 2e n +-logfi (127) 
n * — ' n 

i=l 

where 

(a) follows from the fact that I{W 2 ;Y 2i \Y^~ l) ) < I(W 2 , Y 2 { ^ 1] ; Y 2i ). 

(b) from the fact that H(Y?) + £," =1 ^"i+i)^") = £*=i H ( Y ^' and 013 ■ 

To complete the proof, let us define a new random variable Q uniformly distributed over the set Q = {1, 2, . . . , n} 
with probability ^ as shown in j2U ^3- By defining V = Vq, Y\ = Yiq, Y 2 = Y 2 q, it can be shown that the 
inequalities (jSHJ, and gUJ) are equivalent to (|112|l . (|117|l . and I127|) . respectively. ■ 

E Proof of Lemma [B] 

Let us calculate the probability P(E (1 E% PI E 2 ). If previous error events E\ or £? 2 do not occur, we define other 
error events as follows. 

E 3 : (S n ,T n )£Ai n \ 

E 4 : $m € {1,2,..., 2 nR °} such that (S n , T™, Z"(m)) <E A^ n \S, T, Z), 
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E 5 : $i G {l,2,...,2 nfil } such that S n (i,m) = S n and S n (i,m) G B(m), 
E e : $j G {1,2, . .., 2 nR2 } such that T n {j, m) = T n and T n (j, m) £ C(m), 
Then, 

p(e n El nE%) = P (u 6 i=3 E z n (El n (128) 

6 

< p [e 3 n n £ 2 C )] + ]T p (£i n £ 3 C n (El n £ 2 C )) (129) 

i=4 

By the property of jointly typical sequences [22], H (Ef HEjf)] < | for sufficiently large n. Using arguments 
of Chapter 13 of [221, it can be shown that for sufficiently large n, if i?o > I(S, T; Z) + 62(e) where 62(e) — > as 
e -> 0, then P[E 4 n -Eg n {E{ n J5|)] < f . See Chapter 13 of [22| for a characterization of e 2 . Now 

P[E 5 DEI nE% n (El DEI)] ( "' P[VS n (i,m) G B(m),S n (i,m) ^ S n \S n G A(S'|Z n (m))] (130) 



(b) 
< 



x _ 2 -n(H(S\Z)+e 1 (e)) 



2""i 



(131) 



gexp^C*-*^-^)} (132) 
< I (133) 

for sufficiently large n, if i?i > ff(5|Z) + ei(e) where 

(a) is from the fact that (S n ,T n , Z n (m)) G ^ n) (S,T,Z) implies 5" G A(5|^ n (m)), 

(b) is obtained using the fact that each S n (i, m) has the same chance of equaling S n and independently chosen, and 
the fact that \A(S\Z n (m))\ < 2™W s l z )+ £ i( e )) ; 

(c) follows from (1 — x) n < e~ xn for < x < 1 and n > 0. 
So, 

P[E 5 DE^n (El n El)] = P[E 5 n E% DE 4 n (El n El)] + P[E 5 C\Elr\Elf] (El n El)] (134) 

< P[Ei nEln (El n EQ] + p[e 5 n El n £| n n (135) 

< I (136) 
Similarly it can be shown that P[E 6 r\E$ H (Ef r\E$)] < f if i? 2 > ff(T|Z) + ei(e) for sufficiently large n. Hence 

P[£n(£^nf7S)] < f . ■ 
F Proof of Theorem |U 

Let f(n),g\(n),g-i{n) be a fixed sequence of encoder and decoders. Also, let f(S n ,T n ) = (Wo, W\, W%). We can 
bound the rate Ro as 

nR > H(W ) > I(S n , T n ; W ) (137) 

n 

= yyiiSiWS*- 1 ,!*- 1 ) - H(Si, Ti\ S l -\T l ~\ Wo)] (138) 
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= J2[H(S i ,T i )-H(S i ,T i \Z i )] = J2l(S i ,T i ;Z i ) (139) 

i=l i=l 

Using the constraints on the degrees of the vertexes in the graph associated with messages, we can also have 

n(R 2 + R[) = n(Ri + R' 2 ) > H{Wi, W 2 \W ) - log^ (140) 

= I(S n ,T n ; Wi,W 2 \W ) + H(Wi,W 2 \S n ,T n , W ) - log^i (141) 

( = } I XS n ,T n ;W u W 2 \W )- log /x (142) 

= iT(S n , T"|W ) - H(S n , T n \W u W 2 ,W )- log^ (143) 

> H(S n ,T n \W ) - ne n - log M (144) 

n 

= V Ti| S*- 1 , T*- 1 , Wo) - ne n - log M , (145) 
r-f , ' 

n 

= ^ if (S^ T( | Zi)-ne n - log M , (146) 
i=l 

where (a) is from the constraints on the degrees of the graph, (b) is obtained since W\ and W 2 is a function of S n 
and T n , (c) follows from Fano's inequality. Also, we can also bound the rate R\ as 

nRi > H(Wi) > H(Wi\W ) = I(S n ; Wi\W ) + H(Wi\S n , W ) (147) 

(a) 

> J(S"; Wi\W ) = H(S n \W ) - H{S n \Wi,W Q ) (148) 

(b) n 

> H(S n \W ) - ne n = Y / H(S l \S l -\W ) - ne n , (149) 

i=l 

( c ) ^ 

> J2 HjSil S*- 1 , T*- 1 , Wq ) - ne n , (150) 

n 

= Y J H{S l \Z l )-ne n , (151) 
»=i 

where (a) is obtained since H(W\ |S n , Wo) > 0, (b) follows from Fano's inequality, and (c) follows from adding 
conditioning. 

Similarly, we also can obtain 

n 

nR 2 >Y,H(T l \Z l )-ne n . (152) 

i=l 

Therefore, using arguments similar to those of |36| . we can have the partial converse by dividing the inequalities 
(tbluT . (|H9"j) . (JTSU, and ifTBH) by n, and taking the limit as n — > oo. ■ 
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